System and method for analyzing and displaying computed tomography data

ABSTRACT

A system ( 10 ) and method are provided for analyzing and displaying images rendered from data sets resulting from a scan of a patient. The images displayed include both two-dimensional and three-dimensional views of selected portions of a patient&#39;s anatomy, for example, a tubular structure such as a colon.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority to U.S. ProvisionalApplications No. 60/364,298, filed Mar. 14, 2002 and U.S. ProvisionalApplications No. 60/429,827, filed Nov. 27, 2002, the subject matter ofeach being incorporated herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to analysis and display of dataacquired using computed tomography representing various organs of thehuman body. More specifically, the present invention relates to acomputerized system for analyzing data acquired from tubular structuresof the body, for example, the colon or gastro-intestinal tract, anddisplaying that data in a manner providing for rapid evaluation of thetissue structures of tubular structure and the accurate determination ofany abnormalities present.

2. Description of Related Art

In industrialized nations, colorectal cancer is the second leading causeof deaths from malignancy. In the United States, almost 150,000 peopleare determined to have colon cancer annually. Unfortunately, coloncancer also causes approximately 60,000 deaths annually, with only lungcancer causing more deaths among the population of America. What makesthese statistics truly unfortunate is that colon cancers are perhaps oneof the most preventable of cancers because they usually begin as benignpolyps which grow slowly for five to ten years before becomingcancerous. If these polyps are detected and removed, the risk ofdeveloping colon cancer is greatly reduced.

Regrettably, widespread colorectal screening and preventive efforts arehampered by several practical impediments, including limited resources,methodologic inadequacies, and poor patient acceptance leading to poorcompliance. In addition, a fecal occult blood test (FOBT) fails todetect the majority of cancers and pre-cancerous polyps. Sincesigmoidoscopy only examines a portion of the colon, it also misses manypolyps. The accuracy of barium enema varies among centers and istherefore not always reliable.

Therefore, there is a need for a new test which can be used to screenfor precancerous colon polyps, as well as other abnormalities, such asthe thickening of the colorectal wall caused by tumors that mightotherwise not be found. Like all screening tests, this new test must berelatively inexpensive, minimally invasive, and highly specific.

A technique using helical (also known as spiral) computed tomography(CT) to create computer simulated intraluminal flights through the colonwas proposed as a novel approach for detecting colorectal neoplasms byVining D J, Shifrin R Y, Grishaw E K, Liu K, Gelfand D W, Virtualcolonoscopy (Abst), Radiology Scientific Prgm 1994; 193(P):446. Thistechnique was first described by Vining et al. in an earlier abstract byVining D J, Gelfand D W, Noninvasive colonoscopy using helical CTscanning, 3D reconstruction, and virtual reality (Abst), SGR ScientificProgram, 1994. This technique, referred to as “virtual colonoscopy”,requires a cleansed colon insufflated with air, a helical CT scan ofapproximately 30-60 seconds, and specialized three-dimensional (3D)imaging software to extract and display the mucosal surface. Theresulting endoluminal images generated from the CT scan data aredisplayed to a medical practitioner for diagnostic purposes.

The technique of reformatting 2D cross sections perpendicular to thecolon midline is also described in U.S. Pat. No. 5,458,111, issued Oct.17, 1995 to Coin. However, direct interpretation of the cross-sectionalimages is difficult because a scan of the colon consists of severalhundred axial tomograms. Without advanced image manipulation tools,interpretation of the colon's complex three-dimensional shape from thesecross sections alone is very difficult for a medical practitioner.

One approach to improve accuracy involves production of reformatted 2Dimages at cross sections and orthogonal angles to the colon midline.These reformatted 2D images can provide complimentary diagnosticinformation when viewed with corresponding intraluminal 3D images. Examefficiency can be improved with innovative 3D rendering techniques thatallow fast, interactive evaluation on low priced computer hardware.

While these techniques have advanced the art of using CT scanning inassisting the diagnosis of colorectal cancers, among other diseases, theprior techniques typically required massive computing capacity andspecialized work stations. Moreover, the prior techniques have also beenlimited to providing views of the colorectal pathway that may occludethe presence of polyps, and which do not provide any informationregarding the subsurface morphology of the colorectal pathway, or othertubular structure being analyzed. Typically, the only views provided tothe physician or technician viewing the data display are two andthree-dimensional endoluminal, coronal, sagittal and axial views of thetubular structure being evaluated.

Generally, a CT scanner and computer workstation are used to imagetubular structures of the human body, such as the digestive tract of aliving person. The CT scanner is used to generate cross-sectional axialimages of a human colon which are then transferred to the computerworkstation. A colon midline is defined which follows the colon lumen.The computer workstation supports colon midline definition by generatingand displaying reformatted cross-sectional images, volume renderedscouts, and interluminal views. Semi-automatic midline defining toolsmay also be included. After the midline is defined, a montage of imagesis displayed for diagnostic purposes. The images include axial sections,transluminal cross section, and intraluminal volume rendered images.

Semiautomatic methods are used to delineate the three-dimensional shapeof the colon by identifying its approximate midline. This is supportedby the display of coronal, sagittal, and axial views as well as anoff-axis plane orthogonal to the midline. Straightened curved sectionsalong the midline and a rendered view from the end of the midline mayalso be displayed. Editing the midline in any of these views willtypically cause the other views to update accordingly. After the midlineof the colon is traced, data are extracted so the colon can be examinedefficiently.

In general practice using the equipment described above, a medicalpractitioner, such as a radiologist, examines a tubular structure of apatient, such as the patient's colon, by moving a virtual view pointalong the delineated midline. Three-dimensional orthogonal off-axiscross sections, volume rendered extra-luminal scouts or the originalaxial 2D images, and a high resolution perspective rendering of thecolon's inner surface may be displayed so that the practitioner canobserve the structures of the tissues of tubular structure beinganalyzed. The perspective views can typically be re-oriented in anydirection. A rotatable longitudinal sectioning along the colon's midlinemay also displayed. This view, which may be advanced along the entirelength of the colon, enhances both navigation of the path forward orbackward along the 3D images and interpretation of the 3D images.

Experience has shown that the simultaneous display of cross-sectionaland rendered views enhances diagnostic interpretation of the CT datamore than either cross-sectional or intra-luminal views alone. Volumerendering of the CT data is performed by custom algorithms that usepre-computation of texture and other techniques to achieve interactiveperformance on moderately priced workstations.

Commonly, at least two views of the three-dimensional images generatedas described above are displayed to the practitioner. The first view istypically a forward intra-luminal view which encompasses a view of thecolon from the view point looking away from a terminating location ofthe colon, such as the anal verge. The second view may be a backwardintra-luminal view which encompasses a view of the colon from the viewpoint looking toward a terminating location of the colon, such as theanal verge. Displaying both views makes it less likely that a feature ofinterest will be obscured due to the topology of the colon.

By pointing the cursor on the image and simultaneously pressing a key orotherwise issuing an appropriate instruction to the computer workstation, a practitioner can move the view point off the previouslydefined midline, and all images, including 2D reformatted images, 3Dintra-luminal images, and the 2D axial image may be updated to viewscorresponding to the designated position. The practitioner can randomlymove the view point by using the pointing device, or return to thenearest point on the predetermined midline.

To assist in diagnosis, an unfolded or opened view of the colon may alsobe displayed to the observer. The opened view of the colon correspondsto a view of the entire colon as if it had been physically divided andopened for inspection, just like cutting a garden hose in half along itslongitudinal axis, maintaining the curvature of the colon. However, this“unfolding” technique only allows a view of the internal structure ofthe tubular structure or colon, and does not provide a truly “flattened”view of the internal wall of the colon, wherein the inherent curvatureof the colon is “flattened”, thus allowing structures that protrude fromthe inner wall of the colon, such as Haustral ridges, to hide structuressuch as small polyps or other anomalies from the practitioners view.

Moreover, none of the systems, methods or techniques currently availableare believed to be able to provide an assessment of the wall of thetubular structure. For example, some colorectal cancers do not grow outof polyps, but instead manifest themselves within the walls of thecolon. Such cancers cannot be observed using the means described above,since none of those methods allow for an analysis of the wall thicknessor density of the tubular structure.

What has been needed, and heretofore unavailable, are techniques whichprovide efficient and accurate presentation of CT data that provided aflattened view of a tubular structure that prevents structures withinthe tubular structure from obscuring the view of anomalies such aspolyps in a colon. Moreover, such techniques should be able to runefficiently and quickly on commonly available computer platforms, suchas on computers manufactured by Apple or on IBM clones. Additionally,such a system should be able to also analyze anomalies present in thewall of the tubular structure, such as variations in wall thickness ordensity that may indicate the presence of tumorous growths or otherdisease. The present invention satisfies these and other needs.

SUMMARY OF THE INVENTION

In its broadest aspect, the present invention is embodied in a systemand method of rendering three-dimensional images that provide a userwith views of data generated by a scanner of selected portions of thehuman body that are designed to assist the user, or another person, suchas a surgeon or physician, in diagnosing disease conditions and inprescribing treatment for those conditions. Also disclosed is a systemand method for further processing three-dimensional volume data usingappropriate pattern recognition and analysis software to identifyabnormalities in imaged bodily structures, particularly in tubularstructures, such as the colon, of the body.

In one aspect, the present invention is embodied in a method of imaginga tubular body structure having a lumen defined by a wall, comprisingthe steps of providing a data set containing data representing aplurality of cross-sectional images of a tubular structure of the bodytaken along a longitudinal axis of the tubular body, processing the dataset to reconstruct a three-dimensional image of the tubular body,identifying a central pathway through a lumen of the three-dimensionalimage, selecting a starting point along the central pathway, processingthe data beginning at the starting point and continuing along thelongitudinal length of the three-dimensional image of the colon andrendering a flattened view of the three-dimensional image, storing datarepresenting the flattened view in an image buffer, and displaying theflattened view of the image.

In another aspect, the present invention is embodied in a method thatfurther includes selecting a point along the central pathway, processingthe data at the selected point and rendering an image of a cross-sectionof the wall of the tubular structure at the selected point, anddisplaying the image of the cross-section of the wall of the tubularstructure.

In yet another aspect of the invention, the invention includesprocessing the data and rendering the flattened image, projecting a rayfrom the starting point to the wall, the direction of the raycorresponding to an angle of view, and adding a voxel value to an imagebuffer. In one alternative, the method may also include shifting theangle of view by one degree, and projecting another ray from thestarting point to the wall, the direction of the ray corresponding tothe shifted angle of view if the angle of view has not been shifted atotal of 360 degrees from the initial starting point. In still anotheralternative, the method also includes advancing the starting point alongthe longitudinal axis of the lumen by a selected value if the angle ofview has been shifted a total of 360 degrees from the initial startingpoint, and repeating the steps of projecting the ray and storing voxelvalues in an image buffer until the entire length of the lumen has beenprocessed.

In yet another embodiment, the present invention includes a method forgenerating a view of the interior of a wall of tubular structure of abody including tissue adjacent the exterior of the wall, comprising:

-   -   (a) providing a data set containing data representing a        three-dimensional volume representing a tubular structure of the        body taken along a longitudinal axis of the tubular body, the        tubular body having a lumen defined by a wall;    -   (b) selecting a starting point along a central pathway disposed        along the longitudinal axis of the tubular body;    -   (c) projecting a ray towards the wall a selected distance by        stepping towards the wall along the ray from the starting point;    -   (d) calculating a voxel value at the location of each step of        the ray;    -   (e) adding the voxel value to an image buffer;    -   (f) incrementing the angular projection of the ray one degree;    -   (g) determining if the angular projection of the ray has been        incremented 360 degrees since the starting point was selected;    -   (h) projecting a ray having the incremented angulation toward        the wall;    -   (i) repeating steps (d) through (h) until the angular projection        of the ray has been incremented 360 degrees; and    -   (j) displaying a subsurface volume image.

In another embodiment, the present invention includes comparing athree-dimensional volume data set to a library of geometrical patternsrepresentative of predetermined abnormalities, and identifying astructure contained in the three-dimensional volume data set as abnormalif the structure is determined to match at least one of the library ofgeometrical patterns within a predetermined tolerance. In an alternativeembodiment, the present invention includes further processing theidentified abnormal structure to determine if the identified structureis not abnormal.

In still another aspect, the present invention is embodied in a computerhaving a processor, a program memory and a semi-permanent storagememory. The processor is programmed by a software program designed toreceive data sets generated by a scanner, process the data in the datasets in its memory to render three-dimensional views of the scanneddata, and also render flattened views of the scanned data.Alternatively, the processor processes the scanned data to rendersubsurface views of a structure of interest

Other features and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, which illustrate, by way of example, the featuresof the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graphical representation of a system for taking CT scans ofa patient, including communication means for transmitting the CT scandata to a remote computer for processing and display in accordance withembodiments of the present invention;

FIG. 2 is a block diagram illustrating a generalized process usingembodiments of the present invention for diagnosis;

FIG. 3 is a view of a two-dimensional image of a patient's colon showingidentification of luminal center points of a fly by wire pathway;

FIG. 4 is a view of a representative display showing various views of apatient's colon derived from CT scan data using the methods of presentinvention;

FIG. 5 is a schematic view showing the general processes of a softwareprogram incorporating methods of the present invention;

FIG. 6 is a view of a two-dimensional image of a patient's colonrendered in accordance with the methods of the present invention;

FIG. 7 is a schematic view of a section of a colon showing the drawingof a ray from a luminal center point to the wall of the colon;

FIG. 8 is a side view of a flattened section of the colon section ofFIG. 7;

FIG. 9 is a graphical display showing the relationship of a flattenedsection and an overhead view of a section of the colon;

FIG. 10 is a flowchart representing the steps involved in flattening athree-dimensional image of a colon section;

FIG. 11 is a composite view of a cross-sectional view of a colonincluding points representing the stepping of a ray from a luminalcenter point to and through the colon wall, coordinated with two viewsrepresentative of the display of images rendered using the methods ofthe present invention; and

FIG. 12 is a flowchart representing the steps involved in calculatingand displaying a subsurface view of a section of a colon wall;

FIG. 13 is a flowchart representing an embodiment of the presentinvention that includes computer aided diagnosis performed using patternrecognition software analysis of the stored data generated using aspectsof the present invention;

FIG. 14 is a graphical representation of one embodiment of a report thatmay be generated using the system and methods of the present invention;

FIG. 15 is a graphical representation of another embodiment of a reportgenerated using the system and methods of the present invention that iscoordinated with the embodiment of FIG. 14.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention generally relates to a method and system forgenerating and displaying interactive two- and three-dimensional figuresrepresentative of structures of the human body. The three-dimensionalstructures are in the general form of selected regions of the body and,in particular, body organs with hollow lumens such as colons,tracheobronchial airways, blood vessels, and the like. In accordancewith the method and system of the present invention, interactive,three-dimensional renderings of a selected body organ are generated froma series of acquired two-dimensional images generated from data acquiredby a computerized tomographic scanner, or CAT scan. Recent advances inCAT scanning technology, notably the advent of use of helical or spiralCAT scanners, have revolutionized scanning of human body structuresbecause they allow the collection of a vast amount of data in arelatively short period of time. For example, an entire colorectal scanmay be accomplished in the time a patient can hold his or her breathHowever, as will become clear to those skilled in the art, any type ofdigital image that can be derived from a data set produced by a scanner,such as those produced by CAT scanners, magnetic resonance (MRI), photonemission tomography scanners (PET), ultrasound equipment and the likemay be processed using the system and method of the present invention.

Typically, a patient is located within the scanner and the a scan ismade of a desired portion of the patient's body. The data setrepresentative of the scan of the patient is extracted and stored withappropriate identifying information. The data set may then betransferred to a computer work station for analysis. As is known in theart, the scan data may be compressed so that it can be transmitted to ananalysis site by a variety of means. For example, the present inventionis particularly suited for use on computer work stations based upon apersonal computer, such as a Macintosh computer or IBM clone computerthat have appropriate amounts of working memory, processing power andhard drive storage capacity. The inventors have operated softwareembodying the present invention of a Macintosh G4, sold by AppleComputer, Inc., as well as other computers.

Because the system and method of the present invention does not requirea specialized work station, such as previous methods used to reduce CTscan data and generate three-dimensional images, CT scan data may besent over the internet, local area network or telephone lines usingsuitable communications devices and techniques well known in the art tocomputers and workstations located remotely from the scanner locationoperating under the control of software incorporating aspects of thepresent invention for further processing and analysis.

A patient is initially prepared for imaging by cleansing the patient'scolon. The purpose of the cleansing procedure is to eliminate feces fromthe colon. Optimally, an absolutely clean colon is desired prior to CTscanning, as any retained feces or fluid can simulate or mask smallpolyps.

Once the colon has been cleansed, the colon is insufflated with gas todistend the colon. Distending the colon assures that the interiorsurface of the colon will be clearly visible in the final image display.A rectal catheter, i.e., a standard barium enema catheter (0.5 inch(1.27 cm)) diameter is inserted and gas is introduced into the colon.The colon is filled to a predetermined pressure or volume by introducingthe gas either as discrete puffs or at a constant flow rate. Althoughair can be used, CO₂ may be preferred since CO₂ passes through thecolonic mucosa into the bloodstream and is subsequently exhaled, therebydecreasing the amount of bloating and cramping experienced by a patientafter the examination. Unlike conventional colonoscopy, there is no needto sedate the patient for this procedure.

After insufflation, the colon is then scanned by, for example, a helicalCT scanner to produce a series of two-dimensional images of the colon.The picture elements, or pixels, in the images represent at least onephysical property associated with the colon. The physical property, forexample, may be the x-ray attenuation of the colon wall or of the aircolumn within the colon. The images are generally taken at regularlyspaced locations throughout the abdominal region of the patient. Thesmaller the spacing between successive images, the better the resolutionin the final displayed imagery. Preferably, the spacing betweensuccessive images is approximately 1 mm to produce isotropic voxelssince the X and Y dimensions of each voxel are each approximately 1 mm.

Immediately after inflating the colon, the abdomen is scanned, forexample, with a GE HiSpeed Advantage Helical CT Scanner, during a singlebreath-hold acquisition which is completed in about 30-50 seconds. Thescanning parameters may consist, for example, of a 0.20 inch (5 mm)x-ray beam collimation, 0.4 inch/sec (10 mm/sec) table speed to provide2:1 pitch, and a 0.04 inch (1 mm) image reconstruction interval. Thex-ray beam collimation is the width of the x-ray beam which therebyestablishes the CT slice thickness and Z-axis spatial resolution. Thepitch is the CT table speed divided by the collimation. The imagereconstruction interval reflects the interval at which two-dimensionalimages are reconstructed. Using an x-ray beam collimation of 5 mm and aselected reconstruction interval of 1 mm, images reflecting a 5 mm CTslice thickness are generated at 1 mm intervals. Consequently, there isan overlap of 4 mm between successive 5 mm thick images at a 1 mmreconstruction interval.

A 50 second scan at a table speed of 10 mm per second in a Z-axisdirection creates a volume of data having a Z-axis length of 500 mm.Using a 1 mm reconstruction interval over the length of the Z-axis ofthe volume of data produces 500 images with each image representing a 5mm thick section along the Z-axis. Typically, the scan is done twice,once with the patient in a prone position, and once with a patient in asupine position, thus generating approximately 1000 images. Newermulti-slice CT systems, including those having 4 and 16 channels of dataacquisition bandwidth, can obtain 5 mm thick slices every 2 mm in about30- 40 seconds, yielding data sets having 200-250 images. Scanning 2.5mm thick slices every 1.0 mm increases that number to 400-500 images perdata set. After completion of the CT scan, the rectal catheter isremoved, the gas is expelled, and the patient is discharged.

FIG. 1 generally depicts a system that may be used to render images inaccordance with the principals of the present invention. System 10includes a scanner 12 that is connected to a computer 14. Computer 14 isused to control the operation of the scanner 12, and is also used toacquire data generated by the scanner 12. The computer 14 may be anycomputer having sufficient computational power, storage and memory sothat it is capable of running customized software for controlling thescanner and acquiring and storing the data generated by the scannerduring a scan of a patient's body, or selected portion of a patient'sbody. The computer 14 also includes, besides a processor ormicroprocessor capable of being programmed using custom software, amemory 18 which may be random access memory or read only memory, andsuitable storage media 20, which may include, for example, a hard drive,a CD or DVD drive, or other storage media suitable for storing the datagenerated by the scanner. A display 16 is connected to computer 14 toallow an operator to view displays of data and operating parameters forthe scanner, as well as images rendered from the data generated by thescanner. Additionally, although not shown, computer 14 will also beoperably connected to a device that allows input of commands by the userof the scanner. Such input devices include a mouse or other pointingdevice, a key board, bar code scanner or other device.

The computer 14 may also be operably connected to a modem or othercommunication device that allows provides connectivity to acommunication means 22, such as a telephone line, cable, DSL connection,local area network, the Internet, or a wireless network. This allows thecomputer 14 to communicate with other suitably programmed computersystems or workstations 24, providing for the transmission of datagenerated by the scanner to the other computer systems or workstations24.

Computer or workstation 24 is similarly equipped to communicate viacommunications means 22 with computer 14, providing for communication ofanalysis and reports generated on the computer 24 to a user of computer14. Computer 24 also includes a processor or microprocess, memory 26 andstorage media 26. Additionally, a mouse or other pointing device 32 mayalso be connected, either through a wired or wireless connection, toallow input of commands to the software program running on computer 24to control the operation of aspects of the software program. Theprocessor of computer 24 will typically be programmed with customsoftware incorporating the methods of the present invention to providevarious views of the data generated by the scan which are displayed onthe display 30. While the methods of the present invention will bedescribed below as if the software operating in accordance with thosemethods is controlling computer 24, it will be understood by thoseskilled in the art that the same software may operate and controlcomputer 14, providing a user of computer 14 with the same analysis,views and reports that are generated by computer 24 without the need totransmit data generated during the scan of a patient to a remotelocation.

Referring now to FIG. 2, a generalized flow-chart illustrating the flowof data through a system operating software programmed to carry out themethods of the present invention is depicted. A database containing CTscan data is acquired, either by receiving the database over a wirebased or wireless communication system. Alternatively, the database maybe manually provided to the system, such as through the use of aportably hard drive, Zip disk, CD or DVD or other portable storage mediahaving sufficient capacity.

The scan data typically consists of a series of two-dimensional imagesthat represent slices taken through the structure that was the subjectof the scan. The two-dimensional images are converted or transformedinto a three-dimensional image. The two-dimensional images may betypically obtained from a helical CT scanner operated by a computerconsole. For example, the scanner may be a General Electric HiSpeedAdvantage Helical CT Scanner connected with an optional General ElectricIndependent computer console or physician's console. The computerconsole may, however, be an integral part of the helical CT scannerinstead of a separate independent console. The two-dimensional imagescan also be obtained from ultrasound, positron emission tomography,emission computed tomography, and magnetic resonance imaging. Thephysical property measured is directly associated with the scanningtechnique used to produce the two-dimensional images. For CT images thephysical property measured is typically x-ray attenuation, while formagnetic resonance images (MRI) the physical property measured isgenerally related to various properties such as proton density.

Each of the individual two-dimensional images defines a two-dimensional(X-Y) matrix of picture elements, or pixels, with each pixelrepresenting a predetermined physical property associated with thethree-dimensional structure or organ at a particular location within thethree-dimensional structure. Successive two-dimensional images arespaced in a third-dimensional direction (Z) throughout thethree-dimensional structure.

Typically, the analysis continues by stacking the series oftwo-dimensional images to form a three-dimensional volume 13, therebydefining a three-dimensional matrix (X, Y and Z axes) that represents atleast one physical property associated with the three-dimensionalstructure at coordinates positioned throughout the three-dimensionalvolume. The three-dimensional matrix is composed of three-dimensionalvolume elements, or voxels, which are analogous to two-dimensionalpixels. From the three-dimensional volume, a targeted volume may beselected for three-dimensional rendering. For example, the targetedvolume may include a selected organ or region of interest that is to beisolated from the original three-dimensional volume. The targeted volumemay include an entire organ or, alternatively, may include an air columnor volume confined within an organ or a lumen of the organ.

A dataset representing the original three-dimensional volume may beoptionally subjected to a dataset reduction process to decrease image orspatial resolution or to divide the original volume into smallersubvolumes of data prior to isolating the targeted volume. Datasetreduction and/or subvolume selection are only necessary if thecapabilities of the graphics computer are inadequate to process the fullthree-dimensional volume for effective or efficient three-dimensionalrendering. In order to obtain three-dimensional constructions in realtime, it may be necessary to reduce the size of the dataset representingthe three-dimensional volume of images. For example, the dataset of theoriginal three-dimensional volume may need to be reduced from anoriginal size of 100-250 Megabytes to a reduced size of about 5-10Megabytes. However, the amount of reduction, if any, may vary dependingupon the size of the original dataset and the processing speed andcapability of the computer used for three-dimensional rendering.

Following dataset reduction, the three-dimensional image of the organ orregion of interest is segmented or isolated from the volume of data. Arange of tissue values, which depends on the physical property measured,for example, x-ray attenuation, may be selected to designate the organor region of interest. The organ or region of interest is then isolatedfrom the volume of data. The region of interest may, for example, be theair column comprising the colon or tracheobronchial airways, or someother body part having a lumen filled with a homogeneous substance suchas air, blood, urine, contrast agent and the like. Alternatively, theregion of interest may be a section of bone.

The segmentation process may be effected, for example, by designating arange of physical property values bounded by threshold limits thatfunction to designate the organ of interest. For the selected volume orsubvolume, voxels falling within a selected thresholding range areassigned a single value, for example, 255 to provide a white color,whereas voxels falling outside of the selected thresholding range areassigned a different single value, for example, 0 to provide a blackcolor. Accordingly, the selected volume is thresholded by comparing eachvoxel in the selected volume to the threshold limits and by assigningthe appropriate color value 0 or 255 to each such voxel depending onwhether each such voxel falls inside or outside the threshold rangedefined by the threshold limits. By thresholding the selected volume,the target volume is formed having equal voxel values. Morespecifically, a target organ is produced having a white color while allvolumes outside the threshold limits are produced having a black color.With the exception of various artifacts, which can be eliminated usingimage-processing techniques, only the thresholded target organ iscolored white and everything else is colored black.

Returning again to FIG. 2, once the CT data is acquired in box 50, thedata may be input into a sub-program that segments the data and allows aradiologist or technician to annotate the data set and prepare anoverall report, such as is represented by box 55. This step is optional,and may be omitted without departing from the principals of the presentinvention.

Whether or not the data is processed by the method of box 55, the scandata is modified and output as two-dimensional images, and are stored ina predetermined memory location, such as a folder identified withpertinent patient identifying information, in a suitable file formatsuch as, for example, a “pict” file. Those skilled in the art willunderstand that other file formats may be used, particularly formatsthat are capable of storing the image data while maintaining the fullbit depth of the original axial images while avoiding the necessity ofclipping or compressing the images in a lossy manner.

Once the data is stored in a series of two-dimensional images, the datamay be imported into a sub-program that allows the data to bemanipulated as described above and rendered into three-dimensionalimages, as in box 60. One program that can be used has been developed bythe inventors and titled VISUALIZE, although any rendering program thatcan transform or convert two-dimensional images into three-dimensionalimages having appropriate resolution may be used. For example, therendering process can be accomplished using a general purpose volumevisualization program, such as IRIS Explorer™.

Images of three-dimensional objects, such as the tubular structure of acolon, are projected to two-dimensional screen coordinates fordisplaying the data to an observer using ray-tracing techniques.Imaginary rays sent from a user's viewpoint pass through a viewingplane, referenced to the user's monitor screen, and into the object. Ifa ray intersects an object, the corresponding viewing plane pixel ispainted with a color different from the background color. If nointersection is found, the pixel is painted as background. The criteriaused to stop a ray determines what value is projected onto the viewingplane pixel. This process is different from surface rendering, whichtypically uses polygon rendering to generate an image; in contrast,volume rendering projects a weighted average of all voxels along a rayto generate an image.

The three-dimensional images may now be displayed on a monitor in a formthat provides the user with the ability to “fly” through the tubularstructure, for example, the colon. In order to facilitate the flythrough, and also to allow for more advanced analysis and display of thethree-dimensional images, a central pathway through the colon isdetermined in box 65. This step will be discussed in more detail below.

Once the fly by wire pathway is determined in box 65, the data isfurther processed in box 70 to provide a flattened view of the colon.The flattened view of the colon provides a view that may be likened tocarefully slitting the colon and spreading the colon on a board so thatthe curvature of the colonic wall no longer obscures structures such aspolyps, that may otherwise be hidden when viewed using more traditionalmulti-planar views. Additionally, further processing of the data may beperformed to generate a subsurface view of the wall of the colon,synchronized with the flattened view, to assist in diagnosis of colonicwall abnormalities, such as flat tumors or wall constrictions, thatmight otherwise not be apparent when viewing only the flattened image.

Once all desired image processing is completed, data sets containing theprocessed image data are stored. It will be understood that these datasets contain data arranged in a form that allows a user to view imagesin the data set as if the data set were a movie. This arrangement allowsa physician evaluation the scan data to fly through the colon in anydirection, and to jump to a selected portion of the colon whereabnormalities have been detected and marked for further analysis.

In one embodiment of the present invention, the data sets may betransmitted to a radiologist who receives the flattened volume view, 3Dvolume and subsurface view data for analysis, as indicated in box 75.The radiologist then views the data, and renders his diagnosis of thecase in box 80. The diagnosis report is then sent to a surgeon and/orthe patient's physician in box 85.

Various aspects of the system generally described with reference to FIG.2 will now be described with more specificity. Referring now to FIG. 3,the determination of a central pathway in the colon will now bedescribed. As shown in the figure, a section of the colon, here, thesigmoid colon, has been identified as an area of interest. Center points100 along the lumen of the colon are identified, using eitherspecialized software, or manually by an appropriately trainedtechnician. While automated center “fly by wire” pathway generation maybe used, often manual determination is advantageous to ensure that anaccurate pathway through what is sometimes a tortuous, constricted orcollapsed pathway is obtained. As shown in FIG. 3, a series of pointshave been identified within the central lumen of the colon by atechnician. These points are identified by scrolling through thetwo-dimensional multiplanar representation of the colon and pointing amouse or other suitable device at a desired location and clicking tomark the point in the data base representative of the desired centerpoint. The software memorizes this location and associates it with otherso-determined center points to determine the fly by wire pathway.

The center line of the colon can be determined in one of several ways.For example, in one method of determining the central path through thelumen of the colon, a first point is selected which lies within thelumen of the colon at a desired starting location. The plane through thestarting point that has the minimum cross-sectional area is determinedand the center of such area is marked. A new point is then selectedwhich lies, for example, 1 cm away from center point in a perpendiculardirection relative to the surface area in the plane. A new plane ofminimum area that dissects the colon and passes through the new point isdetermined and the corresponding center of that new area is marked. Thisiterative process is continued until a central path connecting thecenter points is determined.

As stated previously, this is but one of a variety of methods that maybe used to determine the central path through the lumen of the colon,and is not intended to be limiting in any way.

Once the fly by wire pathway has been determined, the data may beprocessed in real time to provide a movie-like fly through of the entirelength of the colon, or selected portion of the colon. Moreover, thesystem and method of the present invention allows the data display to behalted whenever a user desires with an appropriate mouse click or keyboard command. This allows the user to halt the fly through if the userobserves an area of the colon that requires closer evaluation, such asfor the presence of a polyp or tumor. As will be described in moredetail below, the location within the colon where an anomaly isidentified by the user may be displayed to the user in a variety ofways.

Animation of three-dimensional objects, such as that described above asa “fly through”, is achieved by rapidly displaying multiplethree-dimensional views of the volume. Surface rendering calculationsusing presently available computer platforms are fast enough to allowinteractive manipulation of the volume. Additionally, the renderingsoftware typically includes modules that provide the ability to enhancean object's features by altering the shade of color of each viewingplane pixel. For example, “shaded surface” images add an illusion ofdepth by coloring pixels closer to the camera viewpoint with lightershades. Pixels in a shaded surface image reflect the distance betweenthe anatomical structure and the user's viewpoint. Information on therelative “position” of the user and a virtual light source, along withthe information about the rendered structure, are used to appropriatelyshade the rendered volume to produce a realistic impression of theanatomy.

As described, the rendering or converging step 60 occurs rapidly andinteractively, thus giving the user the ability to “fly” through thevolume of data. The direction of “flight” is controlled by using themouse or commands entered through the key board through directionalpointing of the cursor, and the speed (both backwards and forwards) mayalso controlled by pressing buttons on the mouse or key board. The speedof interactive three-dimensional rendering produces a “virtual reality”environment and allows the user to examine the image data in a mannerthat is analogous to real endoscopy.

The path of each simulated flight, whether relative short, or local, ora long distance flight through out the whole or a major portion of thecolon, can be recorded and used in a “playback” mode to retrace theflight path. Individual three-dimensional scenes may also be stored onthe computer like photographs. Each simulated “flight” through the coloncan be recorded on VHS videotape on a video recorder, or on othersuitable storage media, for archival purposes if desired. Additionally,the data may be stored so that it may be transmitted togastroenterologists and surgeons for further review and analysis.

To achieve an adequate speed of rendering but still preserve theanatomical detail of the original CT data volume, it is possible tonavigate or “fly” through reduced three-dimensional imagery built fromthe reduced dataset volume. When the motion stops, however, thethree-dimensional scene may be redisplayed with the highest resolutionpossible using the original CT image volume.

Additionally, it is useful to display other images, such as axial imagesand multiplanar orthogonal axial, sagittal and coronal images during thefly through to assist the user in analyzing the morphology of the colon.An example of such a display is depicted in FIG. 4.

FIG. 4 shows six separate synchronized views of the colon that wererendered using the methods of the present invention. As shown, referenceline 210 points to a particular location within the colon that has beenidentified during the fly through as having structure that requiredfurther evaluation. In this mode of operation, the user flies throughthe colon using the mouse to control the speed and direction of flight.By clicking on the reference line, the images may be advanced or backedup, allowing the user to fly both upstream and downstream. As the userflies through the colon, the reference line moves on the image titled“Variable Width Flattened View”, to be described in more detail below,indicating the location within the colon of the displayed views. Each ofthe other views, such as the Mag Axial, Mag Sagittal (MPR), Endoluminaland Mag Coronal (MPR) views are synchronized with the reference line,allowing a practitioner to study an anomalous structure from a varietyof view points.

Additionally, the display shown in FIG. 4 includes a SubSurface View,which will be described in more detail below. This view allows aphysician or trained technician to also inspect anomalies in the wall ofthe colon. This ability to synchronously display the wall thickness ofthe colon allows for diagnoses of tumors and other disease that may notmanifest as polyps or other anomalous structures that may be seen on theinterior surface of the colon. As shown by the “*” 220 in the SubSurfaceView, and the cross-hairs in the other views, all of the views have beensynchronized to an area of thickened colon wall, providing the user witha variety of views to assess the wall, and to correlate the wallthickness with any other structural abnormalities of the colon.

The subsurface view is capable of being “windowed”, that it, theparameters used to generate the view may be altered as desired by theuser. For example, the available contrast limits may be adjusted tooptimize display of small abnormalities, or the resolution of the viewmay be adjusted. Alternatively, limits on the degree of parameteradjustment or windowing may be programmed into the software to preventhigh contrast, narrow, “lung” setting that are commonly used intwo-dimensional axial, sagittal and coronal polyp searches. If the“lung” windows used are too narrow, a user viewing two-dimensionalmultiplanar views may not observe a constricting tumor if the user failsto adjust the windowing parameters to evaluate the thickness of the wallof the colon.

General System Description

FIG. 5 is a flow chart which sets forth in general the hierarchicalstructure and flow of information through the system and method of thepresent invention. System 300 begins, as is typical for softwareprograms, by calling and initializing various variables, objects, lookup tables and other user definable sub-processes in box 310. After theinitialization is completed in box 310, the monitor displays a defaultwindow layout in step 320. Main loop 330 may then be initialized, andawaits user input to determine which portion of the image to bedisplayed is rendered on the monitor. As previously described, once auser is presented with an overview of the beginning portion of thecolon, the user may use a mouse, or keyboard command, or anotherpointing device, to generate signals that direct the computer upon whichthe software is operating to scroll through the stackedthree-dimensional images that form the three-dimensional volume to bedisplayed. As the user inputs commands into the computer to scrollthrough the data, various sub-programs are initiated, such assub-program 350, which opens the three-dimensional volume and provides apre-flattened large axial multi-planar window layout in step 360. In oneembodiment of the present invention, this step provides rapid renderingof two-dimensional multiplanar images which may be used to by the userto manually designate center points of the luminal image. Alternatively,the data comprising these views may be analyzed using software designedto automatically determine and designate luminal center points, andautomatically advance, slice by slice, through the entire length of thecolonic image, or a designated portion of the colonic image, todetermine the fly by wire pathway through the colon.

Additionally, as indicated by box 370, and as described with referenceto the multiple view display depicted in FIG. 4, the software embodyingthe present invention performs the calculations necessary to flatten thesegment of the tubular structure being rendered and creates a flattenedvolume in box 380, and then displays the flattened view in step 390.While the various views are being calculated, an overview of the colonmay also be generated in steps 400 through 420 and displayed along sidethe various other views.

Once the scrolling or the fly through of the colon is completed, box 430provides for further analysis of the flattened view to identifyabnormalities in the colon. For example, when a user observes anabnormality in a particular flattened section of the colon, thatabnormality may be marked as in box 440, and measured as in box 450.Additionally, local voxel density measurements may be made of asuspected abnormality to attempt to distinguish the suspectedabnormality from, for example, fecal material or an air filleddiverticulum. These measurements may be recorded for further analysis,and the location of the abnormality, established by marking theabnormality in box 440, may also be stored, or provided in a menu ofmarked abnormalities or provided as a print out for further analysis bya user.

FIG. 5 also depicts the various sub-processes that can be undertakenfrom the multi-planar (MPR) views that are displayed in FIG. 4. Forexample, if an abnormality in a particular MPR view is observed, it maybe marked as in box 470. Additionally, as the fly through of the colonproceeds, the abnormality may be tracked to determine its extent andgeneral shape, as indicated in box 480. The images provided to the usercomprise, in essence, an endoluminal movie created along the fly by wirepathway from one end of the colon, for example, the rectum, to the otherend, or cecum, of the colon. The movie may also run in reverse to allowa flight from the cecum to the rectum. In the endoluminal views producedby the present invention, the fly through includes a pointer that can belocked onto the location of a abnormality by the user, and that allowsthe movie to be played in order to display various aspects of theabnormality, assisting in location, identification, observation anddiagnosis of the abnormality.

The various MPR views also provide the user with the ability to alterthe windowing of the view, that is, adjusting the size of the window sothat abnormalities or features of interest of various size may beincluded within a single window, in a sort of zoom in or zoom outfunction, as in box 490. Moreover, the position of the multi-planar viewmay be adjusted as necessary to ensure that to the extent possible, anyabnormalities that might be hidden by structures such as Haustral ridgeswill be visible, as indicated by box 500. The MPR views may also includethe ability to show a 3-D view of the colon, as indicated by box 510,may be windowed for contrast adjustments, and may be used to trigger thefly over local movie, described above, to localize the viewing ofselected abnormalities.

Flattening Module

The present invention includes a method for “flattening” the renderedvolume representative of a tubular structure. Previous attempts atproviding a flattened view have used various methods to display alongitudinal section of the colon, such as would be obtained by slicingthough a chord of the tube. The disadvantage of these methods is thatthe overall view of the colon remained curved, unless such a smallchordal diameter for the section was used as to provide no useful data.In contrast, the inventors have developed a method of truly flatteningthe image of the colon in a manner that may be likened to making asingle slice through the wall of the colon and unrolling the tubularstructure of the colon until it lies flat. By flattening the colon inthis manner, surface abnormalities are highlighted and made visible,even if previously hidden by structures of the colon, such as Haustralridges.

FIG. 6 depicts view of a colon 600 showing a fly by wire pathway throughthe luminal center of the colon 605. Also shown in FIG. 6 are colonsegments 610 and 620, which represent thin slices of the colon which maybe converted from a three-dimensional tubular structure into a flattenedview in accordance with principles of the present invention.

FIGS. 7-9 illustrate the general principles of the flattening process.As shown in FIG. 7, the colon 650 is a tubular structure having aninterior and an exterior defined by a wall. For the purposes ofexplanation only, the tubular structure of the colon can be thought ofas a wall that begins at position zero (not shown) that extendsclockwise around FIG. 7 for 360 degrees.

FIG. 8 depicts a slice of colon wall 655 that has been taken at adesired location. As shown in FIG. 8 the flattened colon wall extendsfrom zero degrees to 360 degrees; the flattened image is formed byunwrapping the wall of the colon (FIG. 7). The width of slice 655 may beany width desired by the user, but typically represents a 1 mm widesection of the colon.

FIG. 9 shows this relationship in more detail. A slice 680 of the colonis converted into a flattened image 685. The edge of polyp “P” justintrudes upon slice 685, and can be seen from a view that sets theobservers viewpoint directly overhead. This allows smaller structuressuch as polyps or other abnormalities to be viewed when they otherwisewould be hidden by protruding structures of the colon, such as Haustralridges.

The process used to render flattened slices of the colon will now bedescribed with references to FIGS. 7-10. As described previously, thefirst step in the overall display process is loading a three-dimensionalvolume or a series of two-dimensional images that are used to generate athree-dimensional volume into the memory of the computer or processorthat is operating in accordance with a software program embodying theprinciples of the present invention. As described above with referenceto FIG. 3, using the multi-planar views of the data set that aredisplayed by the program, a user of the program defines a startingpoint, typically near the anus or rectal area, of a pathway that extendsthrough the colon, defining a fly by wire pathway through the luminalcenter of the colon. The identified points may also be used to identifykey frames, which may be stacked to form the three-dimensional volume.In some embodiments, the program includes a sub-process or sub-processesthat automatically define a directional vector and curved pathwaybetween the point. In other embodiments, the program provides the userwith the ability to automatically center the fly by wire point and toautomatically advance the position along the fly by wire pathway. Oncethe fly by wire pathway has been completed and saved, the same data maybe further processed to create a flattened volume. Alternatively,additional processing of the data may be performed to fine tune,optimize and/or smooth the fly by wire pathway.

The flattening process typically begins with a pre-sampling analysis ofthe data in order to determine the length of the fly by wire pathway,from which the height of the flattened image may be determined. Thispre-sampling process is not absolutely necessary, and may be omittedwithout departing from the principles of the present invention. Thepre-sampling process moves the camera/user viewpoint along the fly bywire pathway. At each point or step along the pathway, the camera isrotated 90 degrees about the Y-axis to face the colon wall. Next, thesample is rotated 360 degrees about the original Z-axis, typically in 5degree increments during the pre-sampling round to reduce analysis time.At each step of rotation, a ray is projected into the colon wall, andthe distance to the wall is measured. The average distance measured isused to determine the colon radius at the particular point in the fly bywire pathway. The distance along the fly by wire pathway is thenincremented, and the process is repeated.

After the pre-sampling analysis, an image buffer may be created. Theheight of the image buffer is determined by the length of the fly bywire pathway through the colon. The width of the image buffer istypically 360 to account for the full 360 degree rotation used toretrace the colon walls.

After the image buffer has been created, the process may be continued togenerate the final flattened image. The flattened final image is createdby once again stepping through the fly by wire pathway. At each stepalong the fly by wire pathway, the camera is rotated 90 degrees aboutthe Y-axis to face the colon wall. Next, the camera (or virtualobserver's viewpoint) is rotated 180 degrees, in 1 degree incrementsduring final flattened image generation, about the original Z-axis. Ateach rotation a ray is projected into the colon wall. The ray tracing orprojection process occurs by incrementing the camera position byapproximately one voxel along the z-vector towards the lumen wall. Whilea one voxel step is described here, steps of other lengths, or multiplevoxels, may also be used. In each step, the opacity of the given voxelis determined by comparing the value of the voxel found at the positionof the step with an opacity table that is defined by the user. If thevoxel that is being stepped through is not air, which is typicallydefined as zero in the opacity table, its value is added to a voxelvalue counter based on the weight of its opacity, relative to apre-defined tolerance. It will be understood that while some voxels aretransparent, they affect the image, but have very little relativeweight. Once the pre-defined tolerance, which is determined by the userdepending on the resolution and contrast, or other windowing parametersselected to optimize the image, has been exceeded, the cumulative voxelvalue is written to the flattened image buffer.

Typically, the first voxel drawn uses the last vertical line in theflattened image buffer. In this case, the image buffer y offset is theheight of the flattened image buffer minus one. The image buffer xoffset would then be 180. As shown in FIG. 9, the flattened image isactually drawn into the image buffer starting at zero degrees androtating to 180 degrees. When a 180 degree rotation has been completed,the camera is reset to its original viewpoint facing the colon wall andthe process is repeated by rotating the camera 180 degrees in theopposite direction.

The image buffer offset is reset to 180 and incremented at each rotationsince the right side of that horizontal line of the image is drawnfirst. When the first horizontal line in the flattened image buffer hasbeen drawn, the y-offset of the flattened image buffer is decremented todraw the next row down. The process is continued, stepping forward alongthe fly by wire pathway and repeating the flattening process at eachincremental step along the fly by wire pathway until a 10 compositeflattened view of has been drawn. Once the flattened image is completed,it is stored to disk and may then be displayed in a separate window, asdepicted in FIG. 4.

FIG. 10 provides a generalized flow chart illustrating the abovedescribed flattening process. As stated previously, the first step inthe flattening process is to initialize variables and set up appropriatebuffers and look up tables that are used by the process, as indicated inbox 710. The process of initializing variables in box 710 also includespicking a starting point along the fly by wire pathway.

Once variables have been initialized and the starting point of the flyby wire pathway has been selected, the process increments one stepforward along the fly by wire pathway. The length of the incrementalstep is user definable, and is generally in the range of 1.0-2.0 mm.Once the first step forward along the fly by wire pathway has beenaccomplished in box 720, the flattened frame sub-process 730 isexecuted.

As previously stated, the camera, which represents the user's viewpoint,is rotated one degree in box 740. A ray is projected to the lumen wallin box 750, as is shown by ray 652 in FIG. 7. (While ray 652 is shownpointing to 360 degrees in FIG. 7, the ray pointing to 1 degree, theposition of the first increment, has been omitted for clarity, but itsposition will be readily understood by a skilled person).

Once the ray has been projected to the lumen wall, a voxel value isadded to a flattened image buffer, as in box 760. As previouslydescribed, the processor tests the opacity value of the voxel along thelength of the ray until it determines an opacity value that is greaterthan zero, indicating that the ray has encountered the inner surface ofthe colon wall.

Once the voxel value has been added to the flattened image, as in box760, the processor determines whether the camera has been rotated 360degrees. If the camera has not been rotated 360 degrees, the programbranches back to box 740, where the camera is rotated one degree and theprocesses of boxes 750 and 760 are repeated. If the camera has beenrotated 360 degrees, the flattened frame process has been completed forthat point on the fly by wire pathway and the process branches back tobox 780. In box 780, the program determines whether or not the end ofthe fly by wire pathway has been reached. If the end of the fly by wirepathway has not been reached, the process branches back to 720, whereinthe location along the fly by wire pathway is incremented one unit(which may be, for example, 1.0 mm). The process is repeated for eachincremental step until the entire length, or selected portion, or thefly by wire pathway, has been processed. Once the incremental stepforward has been executed in box 720, the process then encounters theflattened frame sub-process 730 again, and repeats that as necessary. Ifit is determined that the end of the fly by wire pathway has beenreached in box 780, the flattened image is stored to disk or otherappropriate storage media in box 790, and then the flattened image isdisplayed in box 800. An example of the display of the flattened imagesynchronized with other views of the colon is depicted in the viewtitled “Variable Width Flattened View” shown in FIG. 4. It will beunderstood that the file containing the flattened view data may beplayed like a movie, that is it may be displayed showing forward orbackward flight through the virtual colon, to the provide the user withthe sensation of flying over the surface of the colon, much like anaircraft flight simulation program provides the sensation of flying overa simulated terrain. In the case of the present invention, the terrainis the surface of the colon.

Each horizontal line in a flattened image is linked to athree-dimensional position in the acquired colon volume data set. Sincethe flattened image was created by stepping along a fly by wire pathwaythrough the volume, the user can jump back to the exact place in thevolume from a given position in a flattened image. Vertical position(from rectum to cecum) in the flattened image provides thethree-dimensional position along the fly by wire pathway in the centerof the lumen of the colon.

Using a horizontal position from the flattened image, athree-dimensional position on the lumen wall may be obtained. This valuemay be used to rotate the camera to the angle that was used to project aray to create that pixel in the flattened image. From this position, aray is projected just as in the flattening process to move the camera tothe actual position in the three-dimensional volume. Projecting the rayin this manner, a sub-surface view of the colon wall may be generated byallowing the ray to continue to increment into the colon wall throughsolid tissue.

Generating a sub-surface view of the colon wall in this manner allowsfor the assessment of mucosal and submucosal wall thickness for areas ofabnormality. For example, such a localized abnormality may be one of theroughly 7 percent of colorectal cancers that arise not from slow growingvisible polyps, but from fast growing, aggressive flat tumors which aredifficult to detect by any screening procedure. Moreover, assessing wallthickness in a manner independent of prior window width and window levelsettings, minimizes the possibility for missing constricting tumorswhich can be overlooked using only three-dimensional endoluminal viewsor two-dimensional multiplanar views that rely on narrow high contrastwindow settings to view and analyze CT image data.

Where narrowed colonic segments are observed, the cause may be eitherinadequate air insufflation, spasm, or some structure compressing thelumen either intrinsic to the colon wall, such as inflammatory boweldisease or a tumor, or extrinsic to the colon, such as an adjacent organor mass in the immediate pericolonic vicinity. Using typical “lung”window settings, it is very difficult, if not impossible, for anobserver to determine whether or not the narrowing is due to a standingartifact, such as inadequate air insufflation, or some underlyingdisease or structural abnormality.

The advantages of this approach readily seen with reference to thevariable width flattened view depicted in FIG. 4. Here, reference line210 is set in a portion of a section of the colon that has beenrestricted by a circumferential tumor. As will be apparent to thoseskilled in the art of reading images of the sort depicted in FIG. 4, thecircumferential tumor would not readily be seen from thethree-dimensional endoluminal view present in FIG. 4. If the mag axial,mag sagittal, or mag coronal images also shown in FIG. 4 had beenpresented using traditional “lung” windows (in FIG. 4 these images areproperly windowed) these the constricting tumor would also have beenmissed. However, the sub-surface view, shows a thickened rind of colonwall that is indicative of an advanced malignant colon tumor.

The process for rendering the sub-surface view of the volume data setwill now be described with reference to FIGS. 11 and 12. FIG. 11includes an illustration of how the sub-surface structure is determinedby stepping the ray through the colon wall and into the pericolonicstructure. Also included in FIG. 11 are a variable width flattened viewwith a sub-surface view of a particular section of the variable widthflattened view. The sub-surface view is keyed to the display of thegraphic illustrating the retracing process.

FIG. 12 is a generalized flow diagram showing the various steps includedin rendering the sub-surface view of the volume set for a selected scanline of a flattened image.

Once a user has selected a location within the variable width flattenedview, as indicated by arrow 900 in FIG. 11, the process used to generatethe sub-surface view begins. A ray is projected into the lumen wall, asin box 950. In projecting the ray into the lumen wall, the ray isstepped x voxels along the vector into wall, as in box 970. The programthen determines whether or not the projection has traveled x pixels, thevalue for which is chosen by the user, in box 980. If the projection hasnot traveled x pixels, the process branches back to box 960, and isrepeated. If it is determined that the projection has traveled x pixels,for example, 90 pixels, the program branches to box 990, where thecamera (or viewpoint) is rotated or shifted one degree.

A pictorial example of this process is shown better in FIG. 11. Here,the starting point of the ray is depicted by point A where y, thedistance along the Y-axis, is arbitrarily set equal to zero. This pointis typically selected to be the center of the air filled colonicallumen. The processor then projects the ray outwards towards the colonwall. In general, the ray will be projected from the center point adistance of typically 90 pixels, although other projection distances maybe selected by the user for this analysis. Projecting the ray 90 pixels,or some other desired greater distance, results in the ray projectinginto the wall, allowing the internal structure of the wall, and alsostructures on both sides of the wall, to be imaged. As the ray isprojected, voxel values are calculated based upon an opacity table inbox 990 for any structures encountered by the ray and are added to thesub-surface image buffer in step 1000.

As shown in the diagram in FIG. 11, the ray continues to beincrementally projected to point B, where y equals 35, then to point Cwhere y equals 65. It should be understood that the units of y may notbe an actual measurement of distance, although that is possible.Alternatively, the units of y may indicate a proportional stepdetermined from desired analysis distance starting from the center pointand extending into the wall a desired distance.

At position C where y equals 65, the ray encounters the wall of thecolon. The ray is then allowed to project a further distance, to pointD, where y equals 100, thus rendering an image that includes not onlythe thickness of the colon wall, but also some distance into thepericolonic vicinity.

This stepwise process is also observable in the sub-surface viewdepicted in FIG. 11. In the sub-surface view, point A represents thestarting point in the center of the air filled colonic lumen. Point Crepresents the inner surface of the colon wall. As the ray is projectedto point D in the sub-surface view, the structure of the colon wall aswell as some of the pericolonic structure is viewable. In this manner,an abnormality, located on the surface of the colon wall, indicated byarrow 900, which is otherwise extremely difficult to differentiate fromthe background of the flattened colon can easily be seen in thesub-surface view, as indicated by arrow 905.

The subsurface view, in coordination with the flattened view, providesthe user with two different ways of viewing the colon. The flattenedview may be likened to a stretched aeronautical surface topography mapof the colon that contains surface elevation data of structures, likepolyps, that extend from the wall of the colon. The subsurface view, incontrast, provides a view of the structure above and below the surfaceof the colonic wall. Thus, the sub-surface view generated in accordancewith the principles of the present invention, provides substantialadvantages to the user in providing the ability to observe abnormalitiesor disease related artifacts that would otherwise not be viewable in anyother view presently available for analysis of CT scan data.

Returning again to FIG. 12, after the projection has traveled x pixels,the process returns to box 990, where the camera, or user viewpoint isrotated 1 degree. The process then determines in box 1000 whether thecamera has been rotated 360 times, thus, rotating about the entire innersurface of the colon wall. If the camera has been rotated 360 times, theprogram branches to box 1010 and the sub-surface volume is displayed. Ifthe camera has not been rotated 360 times, the program branches back tobox 950, where the projection of the ray into the lumen wall is repeateduntil the entire image has been rendered.

The isotropic data sets generated using the methods of the presentinvention may also be used as input for a computer aided diagnosticsystem. In such a system, the data sets may be further processed usingpattern recognition software. In this manner, abnormalities may beidentified by computation of three-dimensional features that identifypolyps or other abnormal structure. Various techniques, such as applyingquadratic discriminant analysis, hysteresis thresholding and fuzzyclustering may be used to reduce the occurrence of false positiveidentification of abnormal structures.

FIG. 13 is a flowchart describing in general the steps used to render adiagnosis using pattern recognition to identify structural abnormalitiesin a colon. Using the methods set forth above, the CT scan data isprocessed and three-dimensional volume dates sets are generated in box1110. Data representing the entire colon, or a selected portion of thecolon may be extracted from the three-dimensional volume data set, asindicated in box 1120.

The extracted data set is then processed in box 1130 using a patternrecognition program that has been designed to compare three-dimensionalfeatures of the volume data set with a set of pre-determined geometricalshapes that have been determined to be representative of the geometricshape of abnormalities found in the colon, such as polyps. The programgenerates a listing of abnormalities, including the coordinates withinthe three-dimensional data base where they are located, which may beused to visually assess the abnormality to determine whether theabnormality has been properly identified, or if it is simply astructural feature that resembles an abnormality, but which is otherwisebenign. The data representing the abnormality may also be extracted fromthe data set and stored separately for further analysis.

Although not necessary, it may be desirable to further process, asindicated in box 1140, the three-dimensional data set to further definethe identification of the abnormal structures to reduce the occurrenceof false positives, that is, structures tagged as abnormal but which arein actuality benign. The further processing may be carried on the entiredata set, or an extracted data set containing volume data representativeof the area of the colon surrounding the location of the identifiedabnormality. Such further processing includes applying varioustechniques known in the art, such as quadratic discriminant analysis,hysteresis thresholding and fuzzy clustering to the data to improve theidentification of abnormalities and reduce the occurrence of falsepositives.

Once all processing is completed, a report is generated in box 1150which is communicated to a surgeon and/or the patient's physician. Thisreport may contain various information concerning the scan and theabnormalities identified during the scan, and is intended to aid thephysician in diagnosis and treatment of the patient.

An example of such a report is shown in FIGS. 14 and 15. FIG. 14 depictsa view of entire colon of a patient, with particular segmentsidentified. FIG. 15 shows a computer generated report keyed to thesegments identified in FIG. 14. These reports are suitable for use by aphysician, and may also be given directly to a patient by the physicianto assist the patient's understanding of his or her condition, and anytreatment that may be suggested by the physician.

A particularly advantageous application of the system and methodsvarious embodiments of the present invention is the ability toautomatically, or semi- automatically, render volume data sets, diagnoseabnormalities and generate reports using relatively inexpensivecomputers and displays, as well as the ability to do so using data thatis communicated to a remote analysis site from one or more scanningcenters. For example, one embodiment of the present inventioncontemplates a centralized system or analyzing large volumes of CT scandata communicated to a central analysis location. Such a system wouldallow the scanning centers to scan patients without being burdened withthe overhead of having to also analyze the data and generate reports.For example, a single analysis center using the system and method of thepresent invention would receive and analyze CT scan data from 100 ormore scanning centers.

Once the data is received by the analysis center, the data is eitherprocessed by the center, or alternatively, could be sent to secondarycenters for processing. If the data is sent to secondary centers, thereports and/or processed volume data sets generated by the secondarycenter are sent back to the analysis center for communication either tothe originating scanning center or to surgeons and/or physiciansidentified by the scanning center. Such a system also provides foranalysis of the data using skilled technologists who report theirfindings to radiologists, who then may provide a report containing theiropinion after reading the scan data and their suggestions for treatmentwhich is then communicated to the appropriate surgeon or physician.

It will be understood by those skilled in the art that while the systemand methods of the present invention have been described with referenceto the imaging of a patient's colon, the various embodiments of thepresent invention are equally applicable to the scanning and analysis ofother structures of the body. For example, the flattening and subsurfacemethods described are equally useful to image and analyze any tubularstructure of the body, such as the trachea, esophagus and the like.

While specific embodiments of the invention have been illustrated anddescribed, it will be apparent that various modifications can be madewithout departing from the spirit and scope of the invention.Accordingly, it is therefore intended that the invention be limited,except as by the appended claims./*--------------------------------*/ 1) Rotate to Coronal Anterior Viewfor start gCameraViewAngle.x = 90; gCameraViewAngle.y = 0;gCameraViewAngle.z = 0; 2) Then rotate 360 degrees about z-axis samplingvoxels as we go 3) Next step forward 1 voxel and repeat step 2 */char FlattenFrameNEW( ) {long maxDegrees, z, i; long voxelValue;Point3dDouble tempPosition, startPosition; Uint32 *zOffsetTable; Uint32*yOffsetTable; Uint8 *volBuffer; Uint8 *sliceBuffer; C3dVolume *volume;Point3dDouble tempRotAngle, theRotAngle; if(gUseVariableWidthFlattening) return FlattenFrameVW( ); // Get someglobals from the flattened volume volume = gFlattenedView−>f3dVolume; if(volume != NULL) { volBuffer = volume−>f3dVolumeBuffer; zOffsetTable =(Uint32*) volume−>fZoffsetTable; yOffsetTable = (Uint32*)volume−>fYoffsetTable; gDestVolIndex.x = 359; //Reset the Vectors;Point3dLongToDouble(&gxVectorDouble,gxVectorLong);Point3dLongToDouble(&gyVectorDouble,gyVectorLong);Point3dLongToDouble(&gzVectorDouble,gzVectorLong); ///////////theRotAngle = gzVectorDouble; gzVectorDouble = gxVectorDouble;gxVectorDouble = theRotAngle; theRotAngle.x = −gxVectorDouble.x;theRotAngle.y = gxVectorDouble.y; theRotAngle.z = −gxVectorDouble.z;tempRotAngle = theRotAngle; maxDegrees = 180; SaveViewVectorsDouble( );gDestVolIndex.x = 180; // Start at center for variable width flatteningfor (i = 0; i < maxDegrees; i +=1 ) {Point3dLongToDouble(&startPosition,gCameraPosition); tempPosition =startPosition; voxelValue =SamplePixelFastDouble2(tempPosition,&gzVectorDouble,gSlopFactor,0);Update3dVectorsDoubleNEW(theRotAngle); // rotate 360 degrees aboutz-axis 1 degree at a time gDestVolIndex.x−−;} //**********RestoreViewVectorsDouble( ); theRotAngle.x = −theRotAngle.x;theRotAngle.y = −theRotAngle.y; theRotAngle.z = −theRotAngle.z;gDestVolIndex.x = 180; // Start at center for variable width flatteningfor (i = 0; i < maxDegrees; i +=1 ) {Point3dLongToDouble(&startPosition,gCameraPosition); tempPosition =startPosition; voxelValue =SamplePixelFastDouble2(tempPosition,&gzVectorDouble,gSlopFactor,0);Update3dVectorsDoubleNEW(theRotAngle); // rotate 360 degrees aboutz-axis 1 degree at a time gDestVolIndex.x++; } if (gDestVolIndex.z >= 0){ *(gFlattenRefLineBuffer + gDestVolIndex.z) = gCameraPositionDouble; }gColonFrameIndex++; gDestVolIndex.z−−; return TRUE; }/*-----------------------------------*/long SampleXRayFastDouble2(Point3dDouble tempVectorIndex,Point3dDouble*vectorStep,short slopFactor,long cpuIndex) { Uint8 *theBuffer; long*transparencyValue; long *opacityValue; Uint32 *zOffsetTable; Uint32*yOffsetTable; short theCount; short voxValue; Uint32 volOffset; Uint8theValue; long transpFactor; long grossTranspFactor; Uint32finalIndexValue; Uint32 volumeBufferMax; long volumeOffsetMax; longxCount, yCount, zCount; long tempIndex; Point3dDouble tempVolumeSizeMin,tempVolumeSizeMax; register double tempVectorStepx, tempVectorStepy,tempVectorStepz; Uint8 *overViewImage; Point3dDouble tempPosition,startPosition; char firstVoxel = TRUE; double vertOverSampling;finalIndexValue = 0; grossTranspFactor = 0; theBuffer =g3dVolumeBufferPtr; ////////// zOffsetTable = (Uint32*)gZoffsetTablePtr; yOffsetTable = (Uint32*) gYoffsetTablePtr;transparency Value = (long*) &gTransparencyValue[0]; opacity Value =(long*) &gOpacityValue[0]; #define VERT_OVERSAMPLE #ifdefVERT_OVERSAMPLE ////////////////// // Vertical oversamplingvertOverSampling = (double) gSubSurfaceDepth/128.0; tempVectorStepx =vectorStep−>x * vertOverSampling; tempVectorStepy = vectorStep−>y *vertOverSampling; tempVectorStepz = vectorStep−>z * vertOverSampling;#else tempVectorStepx = vectorStep−>x; tempVectorStepy = vectorStep−>y;tempVectorStepz = vectorStep−>z; #endif volumeBufferMax =gVolumeBufferMax; volumeOffsetMax = g3dVolumeBufferSize;tempVolumeSizeMax = gTempVolumeSizeMax; ///////////////////// // Theloop theCount = min(gViewDepth,200); startPosition = tempVectorIndex; do{ tempIndex = tempVectorIndex.z; volOffset =*(zOffsetTable+(tempIndex)); tempIndex = tempVectorIndex.y; volOffset +=*(yOffsetTable+(tempIndex)); volOffset += tempVectorIndex.x; if(volOffset >= 0 && volOffset < volumeOffsetMax) { theValue =*(theBuffer+volOffset); transpFactor = transparencyValue[the Value]; if(transpFact != 0) { { grossTranspFactor += transpFactor; if(grossTranspFactor >= 0x1000) { if (firstVoxel) { tempPosition =tempVectorIndex; tempPosition.x −= startPosition.x; tempPosition.y −=startPosition.y; tempPosition.z −= startPosition.z; yCount =quickSquareRoot((tempPosition.x*tempPosition.x)+(tempPosition.y*tempPosition.y)+(tempPosition.z*tempPosition.z)); firstVoxel = FALSE; } elseyCount++; if(yCount < 0 ∥ yCount >= 128) // VERT_OVERSAMPLE { //DebugThis(4812); theCount = 0; break; } overViewImage = gXRayImage;overViewImage += yCount * gXRayRowBytes; overViewImage +=gDestVolIndex.x; *overViewImage = theValue; finalIndexValue = 0;grossTranspFactor = 0x10000; //just in case of overflow }finalIndexValue += opacityValue[theValue]; } } } { tempVectorIndex.x +=tempVectorStepx; tempVectorIndex.y += tempVectorStepy; tempVectorIndex.z+= tempVectorStepz; } } while (theCount−− > 0); return((long) theValue); } /*--------------------------------*/ /* Take an “xray” view ofa slice from the flattened image Same code as one flatten sliver (fromone FBW center point) Instead of stopping at surface walls dig below toa certain depth */ char XRayFrame( ) { long maxDegrees, z, i; CMainView*oldMainView; long voxelValue; Point3dDouble tempPosition,startPosition; Uint32 *zOffsetTable; Uint32 *yOffsetTable; Uint8*volBuffer; Uint8 *sliceBuffer; Uint32 volOffset; long rayDistance,tempIndex; C3dVolume *volume; Point3dDouble tempRotAngle, theRotAngle;SaveViewVectors( ); if(!gVirtualColonView−>fVCMovie−>fPlayMovie) {gOverrdeAnimUpdate = TRUE; gMakeFlattenedMovie = 10;gVirtualColonView−>fVCMovie- >FindFrame(*(gFlattenRefLineBuffer +gRefLineNumber)); gMakeFlattenedMovie = FALSE; gOverrideAnimUpdate =FALSE; } // To set view vectors correctly ////////////// // Get someglobals from the flattened volume volume = gFlattenedView−>f3dVolume; if(volume != NULL) { volBuffer = volume−>f3dVolumeBuffer; zOffsetTable =(Uint32*) volume−>fZoffsetTable; yOffsetTable = (Uint32*)volume−>fYoffsetTable; } gDestVolIndex.x = 359; ////////// // Reset thevectors Point3dLongToDouble(&gxVectorDouble,gxVectorLong);Point3dLongToDouble(&gyVectorDouble,gyVectorLong);Point3dLongToDouble(&gzVectorDouble,gzVectorLong); ///////////theRotAngle = gzVectorDouble; gzVectorDouble = gxVectorDouble;gxVectorDouble = theRotAngle; theRotAngle.x = −gxVectorDouble.x;theRotAngle.y = gxVectorDouble.y; theRotAngle.z = −gxVectorDouble.z;tempRotAngle = theRotAngle; Normalize3d(&theRotAngle); maxDegrees = 180;Save View VectorsDouble( ); gDestVolIndex.x = 180; // Start at centerfor variable width flattening for (i = 0; i < maxDegrees; i +=1 ) {Point3dLongToDouble(&startPosition,gCameraPosition); tempPosition =startPosition; voxelValue =SampleXRayFastDouble2(tempPosition,&gzVectorDouble,gSlopFactor,0);Update3dVectorsDoubleNEW(theRotAngle); // rotate 360 degrees aboutz-axis 1 degree at a time gDestVolIndex.x−−; } //**********RestoreViewVectorsDouble( ); theRotAngle.x = −theRotAngle.x;theRotAngle.y = −theRotAngle.y; theRotAngle.z = −theRotAngle.z;gDestVolIndex.x = 180; // Start at center for variable width flatteningfor (i = 0; i < maxDegrees; i +=1 ) {Point3dLongToDouble(&startPosition,gCameraPosition); tempPosition =startPosition; voxelValue =SampleXRayFastDouble2(tempPosition,&gzVectorDouble,gSlopFactor,0);Update3dVectorsDoubleNEW(theRotAngle); // rotate 360 degrees aboutz-axis 1 degree at a time gDestVolIndex.x++; } RestoreViewVectors( );return TRUE; }

1. A method of imaging a tubular body structure having a lumen definedby a wall, comprising: providing a data set containing data representinga plurality of cross-sectional images of a tubular structure of the bodytaken along a longitudinal axis of the tubular body; processing the dataset to reconstruct a three-dimensional image of the tubular body;identifying a central pathway through a lumen of the three-dimensionalimage; selecting a starting point along the central pathway; processingthe data beginning at the starting point and continuing along thelongitudinal length of the three-dimensional image of the colon andrendering a flattened view of the three-dimensional image; storing datarepresenting the flattened view in an image buffer; displaying theflattened view of the image.
 2. The method of claim I, furthercomprising: selecting a point along the central pathway; processing thedata at the selected point and rendering an image of a cross-section ofthe wall of the tubular structure at the selected point; and displayingthe image of the cross-section of the wall of the tubular structure. 3.The method of claim 1, wherein processing the data and rendering theflattened image comprises: projecting a ray from the starting point tothe wall, the direction of the ray corresponding to an angle of view;and adding a voxel value to an image buffer.
 4. The method of claim 3,further comprising: shifting the angle of view by one degree; projectinganother ray from the starting point to the wall, the direction of theray corresponding to the shifted angle of view if the angle of view hasnot been shifted a total of 360 degrees from the initial starting point.5. The method of claim 4, further comprising: advancing the startingpoint along the longitudinal axis of the lumen by a selected value ifthe angle of view has been shifted a total of 360 degrees from theinitial starting point; repeating the steps of projecting the ray andstoring voxel values in an image buffer until the entire length of thelumen has been processed.
 6. The method of claim 4, wherein the tubularstructure is a colon.
 7. A method for generating a view of the interiorof a wall of tubular structure of a body including tissue adjacent theexterior of the wall, comprising the steps of: (a) providing a data setcontaining data representing a three-dimensional volume representing atubular structure of the body taken along a longitudinal axis of thetubular body, the tubular body having a lumen defined by a wall; (b)selecting a starting point along a central pathway disposed along thelongitudinal axis of the tubular body; (c) projecting a ray towards thewall a selected distance by stepping towards the wall along the ray fromthe starting point; (d) calculating a voxel value at the location ofeach step of the ray; (e) adding the voxel value to an image buffer; (f)incrementing the angular projection of the ray one degree; (g)determining if the angular projection of the ray has been incremented360 degrees since the starting point was selected; (h) projecting a rayhaving the incremented angulation toward the wall; (i) repeating steps(d) through (h) until the angular projection of the ray has beenincremented 360 degrees; and (j) displaying a subsurface volume image.8. The method of claim 7, wherein the tubular structure in a colon. 9.The method of claim 1, further comprising: comparing thethree-dimensional volume data set to a library of geometrical patternsrepresentative of predetermined abnormalities; identifying a structurecontained in the three-dimensional volume data set as abnormal if thestructure is determined to match at least one of the library ofgeometrical patterns within a predetermined tolerance.
 10. The method ofclaim 7, further comprising: comparing the three-dimensional volume dataset to a library of geometrical patterns representative of predeterminedabnormalities; identifying a structure contained in thethree-dimensional volume data set as abnormal if the structure isdetermined to match at least one of the library of geometrical patternswithin a predetermined tolerance.
 11. The method of claim 9, furthercomprising: further processing the identified abnormal structure todetermine if the identified structure is not abnormal.
 12. The method ofclaim 10, further comprising: further processing the identified abnormalstructure to determine if the identified structure is not abnormal. 13.A system for displaying views of data generated during a scan of a bodyportion of a patient, comprising: a memory for storing scan data; aprocessor capable of being programmed to analyze scan data, theprocessor in operable communication with the memory and configured tomanipulate data in the memory to render a flattened view of athree-dimensional volume; and a display for displaying the flattenedview.